What speakers do and what listeners look at. A comment on visual deixis and mimesis

In a previous study (Gullberg & Holmqvist 1999),we have shown that listeners’ visual attention isaffected by speakers’ gaze behaviour such thatwhen speakers look at their own gestures, listenerstend to fixate them as well. Visual deixis has oftenbeen offered as an explanation for thisphenomenon. In such explanations it often appearsto be implicitly assumed that listeners automaticallyalign their gaze with that of the speaker. However,empirical data do not support this assumption. Inthis study, we therefore propose a different view ofvisual deixis. We will show that listeners appear tobe sensitive to the difference between speakers’fixations of real and imagined objects. Specifically,we suggest that listeners’ tendency to fixate thesame objects as speakers (i.e. the success of visualdeixis) depends crucially on the different narrativelevels (McNeill & Levy 1982) at which theseobjects occur. These narrative levels can also berelated to the phenomenon of mimesis. The present study is based on 40 video recordingsof conversational narratives. The listeners’ eyemovements were recorded using an SMI iView eyetracker. The preliminary results suggest that thelistener will only fixate speaker fixated objects thatexist in the real world (the paranarrative level). Inother words, if the speaker looks directly at herown, real hand, the listener is likely to fixate it aswell. If, on the other hand, the speaker fixates animagined object which only exists at the narrativelevel (an imagined paper held between the hands),but not at the paranarrative level (where spacebetween the hands is empty), the listener will notfixate that object.REFERENCESGullberg, M. & Holmqvist, K. (1999):Keeping an eye on gestures: Visualperception of gestures in face-to-facecommunication. Pragmatics &Cognition, 7(1): 35-63. McNeill, D. (1992). Hand and mind.Chicago: Chicago University Press. McNeill, D. & Levy, E. (1982):Conceptual representation in languageactivity and gesture. In R. J . Jarvella& W. Klein, eds, Speech, place, andaction. Studies in deixis and relatedtopics. (pp 271-295). Chichester: JohnWiley.